Data Management in a Linear-Array-Based Microscope Slide Scanner

ABSTRACT

Systems and methods for capturing image data using a line scan camera. In an embodiment, a line scan camera captures image data of a sample as a plurality of image stripes. A processor may coarsely align two or more of the plurality of image stripes according to a synchronization process while the line scan camera is capturing at least one of the plurality of image stripes. Subsequently, the processor may also finely align the two or more image stripes using pattern matching.

RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 13/347,077, filed Jan. 10, 2012, which is a continuation ofU.S. patent application Ser. No. 13/114,998, filed May 24, 2011 andissued as U.S. Pat. No. 8,094,902, which is a continuation of U.S.patent application Ser. No. 12/914,356, filed Oct. 28 2010 and issued asU.S. Pat. No. 7,949,168, which is a continuation of U.S. patentapplication Ser. No. 12/235,479, filed Sep. 22 2008 and issued as U.S.Pat. No. 7,826,649, which is a continuation of U.S. patent applicationSer. No. 11/379,648, filed Apr. 21, 2006 and issued as U.S. Pat. No.7,428,324, which is a continuation of U.S. patent application Ser. No.10/371,586, filed Feb. 21, 2003 and issued as U.S. Pat. No. 7,035,478,which is a continuation-in-part of U.S. patent application Ser. No.09/563,437, filed May 2, 2000 and issued as U.S. Pat. No. 6,711,283, allof which are hereby incorporated herein by reference in theirentireties.

BACKGROUND

1. Field of the Invention

The present invention relates generally to the field of virtualmicroscopy and pertains more specifically to data management for verylarge digital imaging files captured by a high resolutionlinear-array-based microscope slide scanner.

2. Related Art

Conventional scanners typically digitize a region of a physical specimenat a desired resolution. As the desired resolution increases, thescanning process becomes more technically challenging. Similarly, thescanning process becomes more challenging as the region of interestincreases or as the available scanning time decreases. Furthermore, theefficiency with which the digitized data can be viewed on a monitor isoften critical to the overall utility of conventional scanningapplications.

Recent technical advances in conventional sensors, computers, storagecapacity, and image management have made it possible to digitize anentire microscope slide at diagnostic resolution, which is particularlydesirable. Diagnostic resolution is the resolution required for atrained technician or clinician to make a diagnosis directly from acomputer monitor, rather than making a diagnosis by looking through theeyepieces of a conventional microscope. Diagnostic resolution varies bysample type, for example, the diagnostic resolution required for a skinbiopsy specimen is typically lower (i.e., diagnosis requires a lowerresolution) than the diagnostic resolution required for other types ofbiopsy specimens.

Although now technically possible, digitizing an entire microscope slideat a diagnostic resolution remains a formidable challenge. Any practicalsolution must capture immense amounts of high quality imagery data in arelatively short amount of time. FIG. 1 is a graph diagram plotting thelimiting resolution in micrometers (“μm”) of an optical system withrealistic condenser settings versus the numerical aperture (“NA”) forthe optical system's microscope objective lens. The limiting resolutionis defined as the smallest distance that can be resolved by the opticalsystem. For example, in an optical system that is designed andmanufactured appropriately, the limiting resolution would be the minimumspatial dimension that can be observed by the human eye.

As shown in the graph, the limiting resolution for an objective lenswith a 0.3 NA is approximately 1.5 μm. Moreover, the limiting resolutionfor an objective lens with a 0.4 NA improves to about 1 μm while thelimiting resolution for an objective lens with a 0.8 NA improves to aneven better 0.5 μm. At this juncture, it is important to note that thelimiting resolution is independent of magnification and depends solelyon the numerical aperture of the objective lens.

Conventional systems that digitize a microscope specimen without losingany details available to the human eye require the dimension of adetector element to be no larger than one half the correspondinglimiting resolution distance. This 2-pixel requirement is based on thewell-known Nyquist sampling theorem. It should be clear that for a2-dimensional imaging system, the 2-pixel requirement translates into anarray of 2 pixels by 2 pixels. Stated differently, if the limitingresolution is 1 μm, then it is necessary to digitize the specimen at 0.5μm per pixel (or better) to capture all of the information that isavailable to the human eye through the objective lens.

FIG. 2 is a graph diagram plotting the scanning resolution in pixels perinch (“ppi”) versus the numerical aperture of an objective lens. Asshown in the graph, an objective lens with a 0.3 NA requires a scanningresolution of at least 38,000 ppi. This resolution is required tocapture all of the details provided by the 0.03 NA objective lens andviewable by the human eye. Similarly, an objective lens with a 0.4 NArequires a scanning resolution of at least 50,000 ppi while an objectivelens with a 0.8 NA requires a scanning resolution of at least 100,000ppi.

FIG. 3 is a graph diagram plotting the scanning resolution in pixels perinch versus the resulting uncompressed file size in megabytes (“MB”) fora one square millimeter (“mm”) region. The graph pertains to regionscaptured as 24-bit pixels (3 color channels, 8-bits per channel). Asillustrated, a 1 mm2 region at 38,000 ppi is approximately 8 MB (ascaptured by an objective lens with a 0.03 NA according to FIG. 2).Similarly, a higher scanning resolution of 50,000 ppi for the same 1 mm2region would result in a file size of 11 MB while a scanning resolutionof 100,000 ppi would result in a file size of approximately 47 MB. Ascan be seen, the size of the image file increases dramatically as therequired scanning resolution, expressed in pixels per inch, increases inrelation to the increasing numerical aperture of the objective lens.Thus, as the scanning resolution increases, the image file sizeincreases significantly.

Accordingly, digitizing an entire microscope slide at a diagnosticresolution results in extremely large data files. For example, a typical15 mm×15 mm slide region at a scanning resolution of 50,000 ppi (i.e.,0.4 NA) would result in a file size of approximately 2.5 gigabytes(“GB”). At a scanning resolution of 100,000 ppi, the resulting file sizequadruples to approximately 10 GB for the same 225 square millimeterarea of a slide.

There are two basic methods that have been developed for scanning entiremicroscope slides: (i) conventional image tiling, and (ii) a novelline-scanning method and system developed by Aperio Technologies, Inc.This latter method utilizes a linear-array detector in conjunction withspecialized optics, as described in U.S. patent application Ser. No.09/563,437, entitled “Fully Automatic Rapid Microscope Slide Scanner,”which is currently being marketed under the name ScanScope®.

Conventional image tiling is a well-known technique. Image tilinginvolves the capture of multiple small, statically sized regions of amicroscope slide using a traditional fixed-area Charge-Coupled-Device(“CCD”) camera, with each capture tile being stored as a separateindividual image file. Subsequently, the various image tiles thatcomprise a specimen are digitally “stitched” together (i.e., alignment)to create a large contiguous digital image of the entire slide.

The number of individual image tiles required to scan a given area of aslide is proportional to the number of pixels that comprise each imagetile. A typical video-format color camera has 768×494 pixels, whichtranslates into 1.1 MB of imagery data per image tile. Recalling that a1 mm2 region of a slide corresponds to 11 MB of imagery data, it followsthat approximately 10 non-overlapping image tiles must be captured todigitize one square millimeter of a slide at a scanning resolution of50,000 ppi. At 100,000 ppi the required number of tiles increasesfour-fold to 40 image tiles per square millimeter.

It follows that for a typical 15 mm×15 mm slide region, at a scanningresolution of 50,000 ppi, a minimum of 2,250 individual image tiles mustbe captured. At a scanning resolution of 100,000 ppi, a minimum of 9,000individual image tiles must be captured. Importantly, each image tilewould have a file size of approximately 1.1 MB. In practice, an evenlarger number of tiles must be captured to provide sufficient overlapbetween adjacent tiles to facilitate the “stitching” together oralignment of adjacent image tiles.

Conventional image tiling systems generally take hours to capture andalign the thousands of tiles required to digitize an entire microscopeslide. Image capture times are significantly increased by the need towait for the CCD camera to stabilize after being repositioned and beforeacquiring an image tile. This wait time is necessary to ensure that thecaptured image does not blur. Practical limitations in data processingspeeds also make the alignment of large numbers of image tiles extremelyslow. In practice, conventional image tiling systems are not able toalign large numbers of tiles without creating “stitch lines” and otherimage artifacts that create computer imaging challenges.

An alternative to image tiling is the afore-mentioned line-scanningmethod. Rather than using a fixed-area camera to capture thousands ofindividual image tiles, the line-scanning method employs a linear-arraydetector in conjunction with a microscope objective lens and otheroptics to capture a small number of contiguous overlapping imagestripes. Unlike the stop-and-go nature of conventional image tiling, themicroscope slide moves continuously and at a constant velocity duringacquisition of an image stripe. One of the many fundamental advantagesof line-scanning over conventional image tiling is that the capture andalignment of a small number of image stripes is significantly moreefficient than the capture and alignment of thousands of separatelycaptured image tiles.

For example, a typical 15 mm×15 mm slide region at 50,000 ppi wouldrequire 15 image stripes, each with a width of 2,000 pixels, todigitally capture the region. Here, each image stripe would have a filesize of approximately 170 MB. At 100,000 ppi, the same region wouldrequire 30 image stripes with each stripe comprising approximately 680MB. The capture of 15 or 30 image stripes for a 15 mm×15 mm area isdramatically more efficient than the capture of 2,250 or 9,000 imagetiles at 50,000 ppi or 100,000 ppi respectively. Furthermore, thecontinuous scanning nature of line-scanning makes it possible to createseamless virtual slides of a region in minutes.

In addition to rapid data capture, line scanning benefits from severaladvantages that ensure consistently superior imagery data. First, it ispossible to adjust the focus of the objective lens from one scan line tothe next, in contrast to image tiling systems that are inherentlylimited to a single focal plane for an entire image tile. Second,because the sensor in a line scanning system is one-dimensional, thereare no optical aberrations along the scanning axis. In an image tilingsystem, the optical aberrations are circularly symmetric about thecenter of the image tile. Third, the linear detector has a one-hundredpercent (100%) fill factor, providing full pixel resolution (8 bits percolor channel), unlike color CCD cameras that lose spatial resolutionbecause color values from non-adjacent pixels are interpolated (e.g.,using a Bayer Mask).

To handle the immense amounts of data produced by conventional imagetiling systems, data management tools have been developed to manage thethousands of relatively small (˜1 MB) image tiles typically generated bysuch systems. These data management utilities, however, are not suitablefor managing a small number of relatively large (˜200 MB) image stripescaptured by the line-scanning image striping system.

Therefore, introduction of the superior image striping system and methodfor digitizing microscope slides has created a need in the industry fora data management system that meets the unique needs imposed by the newtechnology.

SUMMARY

The present invention provides a data management system and method forprocessing and handling the extremely large imagery data files (i.e.,image stripes) that are rapidly produced by a linear-array-basedmicroscope slide scanner. The system receives, processes, and stores thehigh volume of imagery data, which is produced by the linear-array-basedmicroscope slide scanner at approximately 3 GB per minute.

The data are received as a series of coarsely aligned, slightlyoverlapping image stripes that are corrected for image non-uniformitiesand chromatic aberrations and then finely aligned into a seamless andcontiguous baseline image. The baseline image is then logically mappedinto a plurality of regions that are individually addressed tofacilitate viewing and manipulation of the baseline image. Theseplurality of regions are referred to in the industry as “image tiles”but should not be confused with the various separate image files thatare individually captured by a CCD camera in a conventional image tilingsystem.

The data management system enables imagery data compression whilescanning and capturing new image stripes. This advantageously eliminatesthe overhead associated with storing uncompressed image stripes. Theimage compression also creates intermediate level images, therebyorganizing the baseline image into a variable level pyramid structurereferred to as a virtual slide.

The data management system efficiently converts image stripes into ahigh quality virtual slide that allows rapid panning and zooming byimage viewing software. The virtual slide also allows efficientprocessing by an algorithm framework. Furthermore, the functions ofreal-time image processing, compression, and storage are combined withsimultaneous and simplified multi-resolution viewing of high qualityimages at local and remote stations. The data management system is costeffective and scaleable, employs standard image file formats andsupports the use of virtual slides in desirable applications such astelemedicine, telepathology, microscopy education, and the analysis ofhigh value specimens such as tissue arrays.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of the present invention, both as to its structure andoperation, may be gleaned in part by study of the accompanying drawings,in which like reference numerals refer to like parts, and in which:

FIG. 1 is a graph diagram plotting the limiting resolution of an opticalsystem versus the numerical aperture for the optical system's microscopeobjective lens;

FIG. 2 is a graph diagram plotting the scanning resolution in pixels perinch versus the numerical aperture of an objective lens;

FIG. 3 is a graph diagram plotting the scanning resolution in pixels perinch versus the resulting uncompressed file size in megabytes for a onesquare millimeter region;

FIGS. 4A-4B are block diagrams illustrating example microscope slideswith superimposed imagery data stripes according to an embodiment of thepresent invention;

FIG. 5 is a block diagram illustrating an example imagery data stripeaccording to an embodiment of the present invention;

FIG. 6 is a block diagram illustrating an example set of imagery datastripes superimposed on a slide specimen according to an embodiment ofthe present invention;

FIGS. 7A-7B are block diagrams illustrating an example set of misalignedimagery data stripes according to an embodiment of the presentinvention;

FIG. 8 is a block diagram illustrating an example misaligned imagerydata stripe prepared for alignment according to an embodiment of thepresent invention;

FIG. 9A is a block diagram illustrating an example pair of imagery datastripes and their alignment according to an embodiment of the presentinvention;

FIG. 9B is a block diagram illustrating an example set of alignedimagery data stripes and the resulting baseline image according to anembodiment of the present invention;

FIG. 10 is a block diagram illustrating an example blank area of a slideaccording to an embodiment of the present invention;

FIG. 11 is a graph diagram plotting the red/green/blue intensity valuesfor sub-stripe rows according to an embodiment of the present invention;

FIG. 12 is a block diagram illustrating an illumination correction tableaccording to an embodiment of the present invention;

FIG. 13 is a block diagram illustrating an example imagery data stripehaving a line of data with various color channels according to anembodiment of the present invention;

FIG. 14 is a block diagram illustrating an example baseline image withquadrants according to an embodiment of the present invention;

FIG. 15 is a flow diagram illustrating an example process for assemblingimagery data stripes into a baseline image according to an embodiment ofthe present invention;

FIG. 16 is a block diagram illustrating an example stripe offset fileaccording to an embodiment of the present invention;

FIG. 17 is a block diagram illustrating an example viewing platform fordigitized slide images according to an embodiment of the presentinvention;

FIG. 18 is a block diagram illustrating an example image file structuredto enable efficient viewing of digitized slide images according to anembodiment of the present invention;

FIG. 19 is a block diagram illustrating an example image compressor forgenerating a digitized slide image according to the present invention;

FIG. 20 is a block diagram illustrating an example system for datamanagement of digitized slide images according to an embodiment of thepresent invention;

FIG. 21 is a block diagram illustrating an exemplary computer systemthat may be used in connection with the various embodiments describedherein;

FIG. 22 is a block diagram of a preferred embodiment of an opticalmicroscopy system according to the present invention; and

FIG. 23 illustrates a manner in which contiguous image strips acquiredby a linear array detector digitizes a portion of a sample according tothe present invention.

DETAILED DESCRIPTION

A linear-array-based microscope scanner system digitizes entiremicroscope slides or a large regions of a microscope slide in a way thatproduces a small number of slightly overlapping image stripes. Thesestripes, although small in number, are very large in size, for examplearound 200 MB per stripe. Because these image stripes are large and areproduced so rapidly by the microscope scanner system (e.g., 3 GB perminute), conventional methods of receiving, processing, and storingdigital image files are inadequate.

Described herein are certain systems and methods that address the uniquedata management challenges created by the new microscope slide scannersystem. For example, extremely accurate and very fast methods forcombining the image stripes into a seamless contiguous image have beendeveloped that allow the stripe alignment process to take place duringdata capture (i.e, a scan). Stripe alignment while scanningadvantageously eliminates the need for the extremely large image stripesto be stored on the hard disk while the remaining stripes are capturedand then later loaded individually into memory for alignment after thescan is completed. Additionally, stripe alignment while scanningadvantageously allows image data compression contemporaneously withscanning (i.e., in real time) and completely eliminates the need tostore uncompressed image stripes. The seamless contiguous image createdby the alignment of image stripes is referred to herein as a baselineimage. The baseline image is further organized into a pyramid structurethat is referred to as a virtual slide.

Additionally, methods for logically indexing the complete baseline image(i.e., after the image stripes comprising the baseline image have beenaligned) have been developed to allow rapid panning and zooming of thevirtual slide by the virtual slide viewing software. These methodsadvantageously allow a technician or clinician to view any portion ofthe virtual slide at various levels of magnification including a lowresolution such as the naked eye might see or a higher diagnosticresolution as required for the technician or clinician to visualize thedetails required to make a diagnosis.

After reading this description it will become apparent to one skilled inthe art how to implement the invention in various alternativeembodiments and alternative applications. However, although variousembodiments of the present invention will be described herein, it isunderstood that these embodiments are presented by way of example only,and not limitation. As such, this detailed description of variousalternative embodiments should not be construed to limit the scope orbreadth of the present invention as set forth in the appended claims.

INTRODUCTION

In a data management system for a linear-array-based microscope slidescanner, certain functions are paramount. These functions include: (1)image processing during image capture; and (2) image data fileorganization. Image processing during image capture includes the mannerin which the image stripes are processed in parallel with, orimmediately following data capture. Particular aspects of imageprocessing include corrections for optical aberrations (e.g, color,uniformity of illumination), the manner in which the overlapping imagestripes are combined (i.e., alignment), and the means of organizing andstoring imagery data to support efficient viewing on a display monitor,for example, the viewing of virtual slides that represent imagery dataorganized into pyramids. Image data file organization includes themanner in which image stripes are compressed and organized for optimalviewing, including rapid panning and zooming when a virtual slide isaccessed over a network.

Preferably, the data management system is optimized for a specificmethod of image capture. With respect to image stripes that aregenerated by a line-scanner, and in particular a linear-array basedmicroscope slide scanner, the data management system preferablyprovides: (1) runtime management of data during data capture; (2)efficient viewing of very large (e.g., gigabyte) image files; (3) robustimage quality; (4) efficient organization of the imagery data into astandard image file format; and (5) cost effectiveness and scalability.

First, it is desirable to manage the line-scanner imagery data in asnear to real-time as possible. This means processing the image stripesas quickly as they are output by the linear array detector, which ispart of the line-scanner. The desire for such efficiencies is driven bythe throughput requirements of laboratories, including anatomicpathology laboratories that on a daily basis process hundreds ofmicroscope slides representing hundreds of gigabytes of imagery data.The challenges of supporting the approximately 3 GB per minute line-scandata rate of Aperio Technologies' current ScanScope® are formidable,especially since it may require several minutes just to write a 3 GBfile to a typical hard drive.

Second, the data management system preferably supports the efficientviewing of virtual slides. Virtual slides can be displayed on a monitorthat is connected to a local computer or on a monitor that is connectedto a remote networked computer. The network, of course, can be a localarea network, a wide area network, or even the ubiquitous Internet. Forexample, in the case of a pathology microscope virtual slide that isviewed remotely, the data management system should support thetelemedicine application generally referred to as telepathology andadditionally support the simultaneous and coordinated viewing of avirtual slide by multiple parties.

Additionally, the virtual slide viewing software preferably supports theviewing of entire digitized microscope slides with greater efficiencythan the conventional viewing of a comparable glass microscope slideunder a microscope. The virtual slide viewing software minimizes thetime required to wait for screen updates, advantageously enabling localviewing as well as the remote viewing of virtual slides across a varietyof networks. Advantageously, standard image file formats such as thetagged image file format (“TIFF”) support rapid random access to imagedata at any desired level of resolution.

Third, the data management system preferably maintains the highestpossible image quality throughout the data management process. The imagestripes generated by the line-scanner are already of high quality (100percent fill factor), and it is preferable that any requiredpre-processing or post-processing operations (e.g., the correction ofoptical and detector non-uniformities) do not unnecessarily degrade theimage quality of the image stripes. Similarly, the data managementsystem should support the lossy or lossless compression of the imagestripes in order to satisfy a variety of end-user needs. Advantageously,lossy image compression approaches, including JPEG2000, yieldhigh-quality results when subjectively viewed by human experts.

Fourth, the data management system preferably supports the efficientapplication of image processing algorithms to an entire virtual slide orto one or multiple selected areas of the virtual slide, at varyinglevels of resolution. Preferably, the virtual slide image file formatsupports rapid sequential access of a virtual slide, or a region of avirtual slide, by programs that implement image processing algorithms.

Finally, the data management system is preferably cost-effective,scaleable, and capable of implementation using off-the-shelf personalcomputers and conventional file servers and networking equipment. Thedata management system is also advantageously applicable to any type ofmicroscopy imagery data captured by a high-resolution line-scanner,regardless of whether the imagery data represents transmitted light,fluorescence, darkfield, interference contrast, reflected light,phase-contrast or data corresponding to other microscope modalities.Furthermore, the data management system is preferably also applicable toline-scan imagery data that is captured from samples that are notmicroscope slides, including, for example, materials such assemiconductors, circuit boards, micro-well plates, and non microscopyimagery data captured from satellites and other types of spaceexploration vehicles.

Image Processing During Capture

FIGS. 4A-B are block diagrams illustrating sample microscope slides 40with superimposed imagery data stripes 20 according to an embodiment ofthe present invention. In both figures, a specimen 48 is shown on themicroscope slide 40. A typical microscope slide 40 has a slide width 42of approximately 75 mm and a slide height 44 of approximately 25 mm. Alabel 46 is usually fastened on one end of the slide and often holds aprinted barcode or other sample-specific information. The scan area 50designates that area of the microscope slide 40 that should be scannedby the line-scanner. Preferably, the rectangular scan area 50 isselected to be just slightly larger than the largest dimensions of thespecimen 48. The width of the scan area 50 is given by the scan areawidth 52, while the height of the scan area 50 is given by the scan areaheight 54.

Turning to FIG. 4A, the orientation of stripe 20 is perpendicular to theslide width 42. The advantage of this orientation is that the size ofeach stripe 20 is smaller. For example, a maximum of 293 MB is needed toscan stripe 20 at 50,000 ppi and a maximum of 586 MB is needed to scanstripe 20 at 100,000 ppi. The disadvantage is a larger number of stripes20. From a data management perspective, the shorter image stripeconfiguration shown in FIG. 4A is preferable, in part because the slidedigitization can be accomplished more efficiently using the 1-2 GBmemory capabilities of currently available off-the-shelf workstations.

A stripe 20 is shown in FIG. 4A to illustrate the relationship betweenthe image stripe 20 and the specimen 48. At a scanning resolution of50,000 ppi, a 2,048 pixel linear array covers a physical dimension ofabout 1 mm. In the case of a scan area width 52 of 20 mm, approximatelytwenty stripes 20 are required to digitize the scan area 50 thatencompasses the specimen 48. One of the advantages of defining arectangular scan area 50 is that each of the stripes 20 has a similarstripe width. More sophisticated definitions of the scan area 50 areclearly possible if one wanted to more precisely match the scan area 50to the physical area of the specimen 48, for example, in the case ofmultiple fine needle biopsies that are placed over the entire slide arealeaving blank space between tissue areas, or in the case of a tissuemicroarray in which hundreds of small tissue cores are arrayed on amicroscope slide.

In the illustrated embodiment in FIG. 4B, the orientation of the stripe20 is parallel to slide width 42. An advantage of this orientation isthat the number of stripes 20 is limited to approximately 25 stripes ata scanning resolution of 50,000 ppi and 50 stripes at a scanningresolution of 100,000 ppi. A disadvantage is that the size of a stripe20 can become quite large, especially if the scan area width 52comprises a large portion of the slide width 42. For example, if thescan area width exceeds 50 mm, the file size for a single stripe 20 canreach up to 1 GB.

FIG. 5 is a block diagram illustrating an image stripe 20 according toan embodiment of the present invention. A stripe 20 is a digital imagewith a width given by stripe width 24 and a height given by stripeheight 26. A sub-stripe 30 is a digital image with a width given bystripe width 24 and a height given by sub-stripe height 32.

A line-scanner typically generates a stripe 20 by capturing an entirestripe (digital image), one line of data 34 at a time. This line of data34 is preferably one pixel wide in each color channel, and has a heightequal to the stripe height 26. The line of data 34 may also be referredto herein as a column of pixels. The line scanner digitally captures astripe by moving a linear-array field of view 22 (e.g., the field ofview resulting from a linear detector in conjunction with a microscopeobjective lens) in a direction of travel 28 with respect to the slide.Undistorted imagery data is obtained by synchronizing the line rate ofthe linear array detector to the velocity of the microscope slide. Forexample, the microscope slide preferably moves underneath the objectivelens of the slide scanner.

Depending on the type of linear-array detector employed, a line-scannercan produce stripes 20 in monochrome or color. In the case of a colorline scanner, a single line of data 34 may actually correspond to threelines of monochrome data (i.e., three columns of pixels), one line foreach of the three color channels (red, green and blue).

For samples such as microscope specimens, the diagnostic resolution istypically 50,000 to 100,000 ppi, or about 100 times higher than thescanning resolution of a conventional document scanner. At theseresolutions, the stripe height 24 corresponds to a physical dimension ofbetween 0.5 mm and 1.0 mm, when a linear-array detector comprising 2,048pixels is used. While it is possible to utilize linear arrays of varyingdimensions, a 2,048 pixel linear array is preferred.

It should be clear that it is possible to capture image stripes havingany height less than or equal to the maximum number of pixels in thelinear array. There are instances when it might be desirable to captureimage stripes having a narrow stripe height 26 (i.e., less than 2,048pixels) and/or a variable stripe height 26, for example, when the tissuetopology is such that one or both edges of a stripe are not perfectlyfocused. Advantageously, the data management system is adaptable toaccommodate these more sophisticated scanning approaches.

In the case of a 2,048 pixel linear-array, each pixel covers a physicalsample distance of 0.5 μm per pixel at 50,000 ppi, and 0.25 μm per pixelat 100,000 ppi. Additional nontrivial optics and focus adjustments arerequired to collect light from such a small physical area of thespecimen and to focus it onto a linear array that, in the case of a2,048 pixel array, measures approximately 28 mm. Preferably, thecaptured imagery data is well focused and has been acquired by aline-scanner that is capable of acquiring the image in stripe format.

For a microscope specimen that measures 25 mm along one dimension, thedimensions of stripe 20 are 1.0 mm by 25 mm at a scanning resolution of0.5 μm per pixel. This translates into a stripe 20 that comprises 2,048pixels by 50,000 pixels. Assuming 24-bit pixels (8-bits for each of red,green, and blue color channels), a single stripe 20 comprises about 102million pixels, or 293 MB of color imagery data. At a higher scanningresolution of 0.25 μm per pixel (i.e., 50,000 ppi), the dimensions ofstripe 20 are 0.5 mm by 25 mm, or 2,048 pixels by 100,000 pixels. Inthis latter case each stripe 20 comprises 205 million pixels, or 586 MBof color imagery data. Multiple stripes 20 are captured by theline-scanner to digitize a typical microscopy specimen, for example, atissue or cytology specimen that may consist of several hundred squaremillimeters of slide area. These multiple stripes 20 are then aligned tocreate a seamless large contiguous image of the entire microscopespecimen.

Preferably, the microscope slide line-scanner can acquire and outputcolor data at 9,000 lines per second using a line-scan camera such asthe Basler L301bc. This camera features a 2,048 pixel linear-array thatcan scan a stripe 20 at 55 MB of color imagery data per second, or 3.3GB per minute. At this data rate, a single stripe 20 with a stripe widthof 25 mm comprises 293 MB and can be scanned in 5.3 seconds. There areseveral reasons why the effective data rate for capturing multiplestripes 20 can be less than 3.3 GB per minute, including (i) delaysassociated with accelerating to and decelerating from the constantvelocity at which image data are captured; (ii) processing delays suchas correcting for non-uniform illumination; (iii) mechanical delaysassociated with physically moving the linear-array detector from onestripe 20 to another; (iv) processing delays associated with aligningadjacent stripes 20; (v) data compression delays, and (vi) delaysassociated with saving the imagery data corresponding to a stripe 20.One advantage of the present invention is to minimize these delays so asto achieve a high effective scanning data rate.

FIG. 6 is a block diagram illustrating an example set of image stripes20 superimposed on a slide specimen according to an embodiment of thepresent invention. The same specimen 48 and scan area 50 describedpreviously are shown, together with several superimposed stripes 20.Optimally, the stripes 20 are perfectly synchronized and aligned duringthe capture process, however in practice, upon capture, the stripes 20may be both misaligned and overlapped.

FIG. 7A is a block diagram illustrating an example set of misalignedimage stripes 20 according to an embodiment of the present invention. Inthe illustrated embodiment, adjacent stripes 20 are offset from oneanother by a coarse stripe offset 56. For example, in a microscope slidescanner that captures 9,000 lines of data per second, a 100 msecdifference in the start of the acquisition of adjacent stripes 20 maycorrespond to a coarse stripe offset 56 of 900 pixels.

Thus, there is a need to coarsely align the stripes 20 along thedirection of travel 28. This coarse alignment, as well as the subsequentfine alignment that is required to provide accurate pixel-to-pixelregistration between adjacent stripes 20, can be accomplished as apost-acquisition operation after all of the stripes 20 that comprise thescan area 50 have been acquired. The disadvantage of such apost-acquisition stripe alignment is that the captured stripes are savedto disk until the entire scan area is captured. Then all the stripes 20have to be read from the hard disk into memory to facilitate coarse andfine alignment. Given the size of the stripes 20, this approach can beextremely time-consuming. Preferably, alignment of adjacent stripes canbe accomplished during the scanning process, while adjacent stripes 20are in memory, and prior to saving the stripes 20 or the virtual slideon the hard disk of a computer or in other persistent memory.

FIG. 7B is a block diagram illustrating an example set of overlappingand mis-aligned image stripes 20 according to an embodiment of thepresent invention. In the illustrated embodiment, the image stripes 20are intentionally overlapped in the axis that is transverse to thedirection of travel 28. This overlapping of image stripes facilitatesthe alignment of adjacent stripes 20. Notably, the size of the stripeoverlap 58 must be sufficient to support the fine alignment of adjacentstripes, and is preferably 40 pixels.

FIG. 8 is a block diagram illustrating an example image stripe 20prepared for alignment according to an embodiment of the presentinvention. The need for aligning the data stripes arises from therealization that data latencies in conventional operating systems andthe slide positioning hardware of typical line-scanners do not make itpossible to begin each stripe at exactly the same location for eachstripe 20. Preferably, an external synchronization method can be used tocoarsely align the stripes 20 during scanning, followed by an iterativefine-alignment process.

For example, the line-scanner preferably provides position feedbackinformation from a position encoder that is mounted either to the motoror to the mechanical stage that is moving the microscope slide duringscanning. This position feedback information, together with priorknowledge of the dimensions and location of the scan area on themicroscope slide, are used to set a coarse alignment synchronizationflag 60 when the line-scanner reaches the beginning of the scan area 50.Due to data latencies in the overall line-scanning system, the coarsealignment uncertainty 68 in issuing the coarse alignment synchronizationflag 60 is approximately +2 milliseconds, which translates into +18lines of data at a scanning rate of 9,000 lines per second. Since theline-scanner must be moving at constant velocity while capturing validline-scan data 66, it is necessary to allow the line-scanner enough timeto accelerate to the desired velocity during a ramp-up period 62. Thetypical ramp-up period corresponds to less than 1 mm of travel of theline-scanner.

In a preferred embodiment, a large line-scan buffer 64 is established assoon as the line-scanner begins its motion. Imagery data are capturedand temporarily stored in the line-scan buffer 64 as the line-scannermoves across the microscope slide 40. At some time during the capture ofthe line-scan buffer 64, the coarse alignment synchronization flag 60 isset. This flag is preferably set using a strobe light that saturates thelinear-array detector in those lines of data of the line-scan buffer 64that correspond to the desired starting point of the valid line-scandata 60.

For example, when using the Basler L301bc detector, three saturatedcolumns (lines) of pixels, each separated by eight pixels andcorresponding to red, green, and blue respectively are clearly apparentin the line-scan buffer 64. Following capture of the entire line-scanbuffer 64, the leading edge of the red column of saturated pixels can beidentified. To eliminate those lines containing saturated pixels, thebeginning of the valid line-scan data 66 can be designated as being thefiftieth column of pixels following the leading column of red saturatedpixels. Knowledge of the dimensions of the scan area makes it possibleto determine how many lines of data comprise the valid line-scan data66, that is, the line-scan imagery data desired in the image stripe 20.Therefore, instead of saving the entire line-scan buffer 64 that hasbeen buffered, all of the columns of pixels that precede the beginningof the valid line-scan data 66, and those that follow the end of thevalid line-scan data 66, are eliminated prior to saving the image stripe20.

The aforementioned use of a strobe light for setting the coarsealignment synchronization flag 60 is particularly useful when the datacapture board that directs the line-scan data to a temporary buffercannot be triggered externally to start or stop the capture of line scandata. In the event that the data capture board supports an externaltrigger to start or stop data capture, an alternate preferred method forcoarsely aligning the image stripes 20 is to initiate the data captureusing an external trigger after a coarse alignment synchronization flag60 has been set. In such an embodiment, the line-scan buffer 64 and thevalid line-scan data 66 are identical because the data in the line-scanbuffer 64 begins with the setting of the synchronization flag 60.Advantageously, no data are captured prior to the coarse alignmentsynchronization flag 60.

Similarly, the data capture board can be triggered again, this time toturn off the capture of valid line-scan data 66, when the desired numberof lines of data comprising the valid line-scan data 66 have beencaptured. It should be clear that it is not necessary to use a strobelight if the data capture board can be triggered externally. A preferredmethod is to use the position encoder output to initiate the externaltrigger to the data capture board.

FIG. 9A is a block diagram illustrating a pair of image stripes andtheir alignment according to an embodiment of the present invention.Overlapping stripes are aligned by pattern matching within the stripeoverlap 58, a region that is preferably 40 pixels wide. The alignmentresults in an X-Y offset 74 for each stripe. The X-Y offset 74 denotesthe exact (x,y) coordinate location in the reference stripe 70 where theadjacent stripe being aligned 72 will be placed. The resulting compositeof the aligned reference stripe 70 and stripe being aligned 72 is alsoshown in FIG. 9A. Advantageously, in the stripe overlap region 58, theimagery data from one stripe is used while the redundant imagery data inthe adjacent stripe can be discarded.

In one embodiment, calculation of the optimal X-Y offset 74 proceeds bytesting a range of possible offset values along both the x-axis (i.e.,the scanning axis that indicates the direction of travel 28) and they-axis (i.e., the axis that is transverse to the direction of travel28). A potential X-Y offset 74 is chosen and the stripe being aligned 72is shifted to the corresponding coordinate location in the referencestripe 70. The pixel intensity values in the reference stripe 70 arethen subtracted from the shifted-pixel values in the stripe beingaligned 72 for all pixels in the overlap region 58 for the color channelexhibiting the highest signal-to-noise ratio. These differences are thensquared to yield a positive number and summed. This sum is a measure ofalignment error for the particular X-Y offset 74 being tested. Aftertesting all possible X-Y offsets 74 in the allowable range, the X-Yoffset 74 pair with the smallest sum-value is chosen as the alignmentX-Y offset 74. This method advantageously finds the point of maximumcorrelation between the two stripes.

The range of x-values to be tested can be determined by the coarsealignment uncertainty that gives rise to different values of the coarsestripe offset 56. The range of y-values to be tested can be determinedby the mechanical motion uncertainties (e.g., position error) fromstripe to stripe. There is no upper limit to the range of values to betested, although the reference stripe 70 and the stripe to be aligned 72need to overlap for the offset (correlation) calculation to besuccessful.

A significant improvement in stripe alignment performance is obtained byselectively including pixels in the region corresponding to the overlapof reference stripe 70 and the stripe being aligned 72 in thecomputation of the error sum that is used for computing the correlationbetween stripes. For example, pixels to be included in the stripealignment calculation can be selected according to the premise thatobjects, such as object 71, that are divided between two adjacentstripes will be aligned when their corresponding edges are aligned.Because objects are 2-dimensional and their corresponding edges are1-dimensional, the edge pixels, which are much smaller in number, arepreferably used for stripe alignment.

Edge pixels can be identified by a large value in the local intensitygradient. For example, the pixels in reference stripe 70 correspondingto stripe overlap 58 are first sorted according to the local intensitygradient value. The sorted list is then used to define a small subset ofpixels from within all the pixels comprising stripe overlap 58 havingthe largest intensity gradient values. This list is then used forcomputing the error sum at each potential X-Y offset 74. In practice, ithas been demonstrated that 2,000 object-edge pixels, from the more than500,000 pixels in a typical stripe overlap 58 are sufficient to yieldaccurate X-Y offsets 74. This significant reduction in the number ofpixels examined correspondingly reduces calculation time by more than afactor of 100, and allows the stripe alignment to be successfullycompleted in a few seconds on a typical personal computer (PC). Thisdrastic reduction in stripe alignment time advantageously allows thestripe alignment to take place during the data capture process, whileadjacent stripes are resident in memory.

FIG. 9B is a block diagram illustrating an example composite image 76and the resulting baseline image 80 according to an embodiment of thepresent invention. Once all the stripes have been aligned and combinedinto a composite image 76, there may be a number of overhanging edges78. These edges 78 can advantageously be cropped from the compositeimage 76 to create the baseline image 80.

It is noteworthy that accurate alignment of adjacent stripes 20 requiresonly linear translations between stripes. No warping or other non-lineartransformations are required to create seamless baseline images. One ofthe underlying reasons for creating seamless imagery data is founded inthe ability of the linear-array-based microscope slide scanner tomaintain constant velocity during scanning. A second reason for theability of a linear scanner to achieve seamless images is consistentlysuperior imagery data, which is a direct and inherent result of datacapture using a line-scanner.

FIG. 10 is a block diagram illustrating an example blank area 92 of amicroscope slide 40 according to an embodiment of the present invention.The blank area 92 is preferably used for illumination correction.Illumination correction is needed to correct the raw imagery data thatis captured by the line-scanner for artifacts such as shading,pixel-to-pixel non-linearities and to compensate for the opacity andrefractive properties of the slide glass, including the effects of themounting media. The blank area 92 has a blank area width 94 thatcomprises an arbitrary number of lines of data 34. A typical value forthe blank area width 94 is 2,000 lines of data. The blank area height 96may correspond to the number of pixels in the linear-array detector. Theblank area height 96 is preferably 2,048 pixels. A blank area row ofdata 102 comprises the intensities measured by the same pixel in each ofthe lines of data 34 that comprise the blank area 92. In the preferredembodiment, there are 2,048 rows of data, starting with a blank areafirst row of data 98, followed by a blank area second row of data 100,and ending with a 2,048th blank area last row of data 104.

Optimally, all of the intensity values in the blank area 92 are the samesince the ideal blank area is entirely uniform and there are nopixel-to-pixel variations between blank area rows of data 102. Inpractice, however, there may be spatial artifacts in the blank area thatare preferably ignored in the computation of any illuminationcorrection. Consequently, there will be pixel-to-pixel variations alonga given blank area row of data 92. Shading and other aberrations canalso contribute to variations in the intensity values along the lengthof the linear-array (i.e., along the blank-area height 96).

FIG. 11 is a graph diagram plotting example values of the red, green andblue intensity values for an image stripe 20 that corresponds to a blankarea 92 according to an embodiment of the present invention. In theillustrated embodiment, the following three-step procedure can be usedto compute the average intensity of each row of pixels in the blank area92: (i) for each color channel, the average of the intensities of eachpixel within a blank area row of data 102 is computed; (ii) any pixelshaving an intensity value that differs by more than 5 counts from theaverage computed in step (i) are eliminated; and (iii) the averagevalues originally computed in step (i) are re-computed without thecontributions of the pixels discarded in step (ii). This procedureprovides a means of excluding debris in the blank area and provides anaverage intensity value for each pixel for each color channel as shownin FIG. 11.

FIG. 12 is a block diagram illustrating an illumination correction table112 according to an embodiment of the present invention. In theillustrated embodiment, the entries in the illumination correction table112 are obtained by calculating, for each pixel and for each colorchannel, the ratio between an intensity value that is defined torepresent white and the average value computed from within the blankarea 92. For example, if the average red intensity for pixel number 603is 203 counts, and white is defined to represent a count of 245 for eachchannel, then the entry in the illumination correction table 112 is245/203, or 1.197. Any data that are subsequently captured are correctedby multiplying the pixel-specific and color-specific entries in theillumination correction table 112 by the actual intensity output by thescanner.

FIG. 13 is a block diagram illustrating an example image stripe 20having a line of data 34 with various color channels according to anembodiment of the present invention. Chromatic aberrations resultbecause light at different wavelengths takes a slightly different paththrough the optical system of the scanner, resulting in shifts ofmultiple pixels between color channels at the outermost edges of astripe. Longer wavelength light (red) will result in slightly widerstripes than blue light. Chromatic aberration correction involvesshifting the pixels of the independent color channels of the line-scanimagery data “inward” from the edges of a stripe 20 by a pre-determinedamount. The amount of pixel shift required to correct for chromaticaberrations is determined empirically. The pre-determined amount ofpixel shifting is a function of the optics of the scanner, and differentoptical designs may be subject to more or less chromatic aberrations.

In the illustrated embodiment, a stripe 20 having a height of 2048pixels is captured by a linear-array based scanner having a direction oftravel 28. The red color channel, illustrated by color channel array278, is divided into various zones of pixels, including: (i) anoutermost Zone A (276) comprising 139 pixels; (ii) an intermediate ZoneB (274) comprising 360 pixels; (iii) an intermediate Zone C (272)comprising 360 pixels; and (iv) a central Zone D (270) comprising 330pixels. Preferably, each zone can be shifted inward by a different,fixed amount of pixels. For example, pixels in zone A are shifted inwardby 3 pixels, pixels in zone B are shifted inward by 2 pixels, pixels inzone C are shifted inward by 1 pixel and pixels in zone D are notshifted.

The shifting of pixels to compensate for chromatic aberrationspreferably results in a stripe 20 that may be slightly narrower than theoriginal imagery data, and any excess pixels at the edge having lessthan three color channels are trimmed off. The amount of pixel shiftingis less for the green color channel where only three zones of pixels aredefined. Pixels in the blue color channel are not shifted at all. Forshorter stripes (e.g., 1000 pixels) the amount of pixel shifting canadvantageously be pro-rated based on the height of the stripe.

FIG. 14 is a block diagram illustrating an example baseline image 80comprising standard image tiles 114 in accordance with the standardtiled TIFF image format. Note that the process of logically organizingimage stripes into standard image tiles 114 refers to the dataorganization of the captured stripes 20, and is unrelated to the methodof image acquisition. In contrast to a conventional image tiling systemwhere individual image tiles are acquired by a CCD camera, standardimage tiles 114 are a well-known method for organizing and manipulatinghigh-resolution images, as will be understood by one having ordinaryskill in the art.

The advantages of tiled TIFF images are well-known. For example, theTIFF Specification, Revision 6.0 (Jun. 3, 1992), which is incorporatedherein by reference in its entirety, discusses the benefits of tilingfor high resolution images. These benefits include more efficient accessto the image and improved compression. Thus, while the most efficientway to capture imagery data can be to acquire the imagery data asstripes using a linear-array-based scanner, there are also significantadvantages to organizing the image stripes, after they have beencaptured, into standard image tiles 114. Advantages of tilizing imagestripes include providing rapid access to sub-regions of the baselineimage 80, supporting rapid panning and zooming by image viewingsoftware, and the processing of image data.

In the illustrated embodiment, a baseline image 80 is shown comprising50,000 pixels (width) by 30,000 pixels (height). Baseline image 80 mayalso comprise a plurality of regions such as display image 250. Forexample, display image 250 may be a region that measures 1,280 pixels by1,024 pixels, which is a typical number of pixels that can be displayedon a standard computer monitor.

One way to store and access the baseline image 80 is to store 30,000separate and discrete stripes that each comprise 1 pixel by 50,000pixels. When attempting to present the display image 250, however, each1 pixel stripe must be read to access the portion of the stripe (if any)that contributes to the display image 250. In this case, 1,024 suchbuffers must be read, with 1,280 pixels being displayed from eachbuffer. In total, 51.2 million pixels (50,000×1,024) must be read, and atotal of 1.3 million pixels are displayed. The ratio between the amountof imagery data that must be read divided by the amount that isdisplayed is 40 (51.2/1.3). This ratio provides a measure of therelative inefficiency of viewing imagery data stored as separate anddiscrete stripes.

An alternative way to store and access the 50,000×30,000 pixel baselineimage 80 is to store the entire image as a single file and logicallydivide the entire image into standard image tiles 114 of, for example,240 pixels by 240 pixels. The result is a single contiguous baselineimage 80 that has been logically divided into standard image tiles 114.It follows that in order to present the display image 250 thatcorresponds to 1,280×1,024 pixels, only the data corresponding to amaximum of 6×5 standard image tiles, or 1,440 pixels by 1,200 pixels(1.7 MB), needs to be read. In such an embodiment, the ratio between theamount of imagery data that must be read divided by the amount that isdisplayed is 1.3, resulting in a substantial improvement when usingstandard image tiles versus separate and discrete image stripes.Advantageously, the TIFF file format and the JPEG2000 compressionstandard support the organization of large baseline images into standardimage tiles 114 in a single convenient file format.

FIG. 15 is a flow diagram illustrating an example process for assemblingimage stripes into a baseline image according to an embodiment of thepresent invention. Initially, in step 200, an image stripe is acquiredby the high-resolution line-scanner. An image stripe is acquired oneline of data (i.e., one column of pixels) at a time. In one embodiment,all of the stripes required to assemble a baseline image are acquiredsequentially. Preferably, the data is captured one line of data at atime and comprises high quality and well-focused imagery data.

The stripe acquisition step preferably employs a synchronization flag toindicate when the line scanner should begin data capture. In oneembodiment, the synchronization flag is a hardware trigger from theposition encoder that is coupled to the mechanical stage that moves themicroscope slide. Employing a coarse alignment technique such as asynchronization flag advantageously ensures that when the line-scannerhas completed the acquisition of one stripe and is ready to acquire thenext stripe, it can begin stripe acquisition at a proper, efficient, andaccurate location.

After stripe acquisition, in step 202 the imagery data is corrected forshading, pixel-to-pixel non-uniformities, and to implement otherdesirable image enhancements, for example, gamma correction. Preferably,illumination correction is applied one line of data at a time until theentire stripe has been corrected. Illumination may also take placeduring data capture.

In one embodiment, an illumination correction reference such as thatpreviously described with respect to FIG. 12 can be used to determinethe pixel-specific, color-channel-specific adjustments that are appliedto the image stripe during illumination correction. Advantageously, theuse of an illumination correction look-up-table is extremely fastbecause the value of an incoming pixel that is part of a stripe issimply exchanged with another value, without the need for more complextime-consuming computations.

Following illumination correction, the stripe is next corrected forchromatic aberrations in step 204. The chromatic aberration correctionprocess is similar to illumination correction in that it also applied tothe imagery data one line of data at a time until the entire stripe hasbeen color corrected. Preferably, chromatic aberration correction cantake place contemporaneously with data capture and illuminationcorrection.

Once the stripe has undergone chromatic aberration correction, thesystem determines, in step 206, if a previously captured adjacent stripeis present. In the case where the captured stripe is the first stripe ofthe baseline image, no adjacent stripe is available and the processreturns to step 200 to acquire another stripe. Where the captured stripeis the second or later stripe of the baseline image, the adjacent stripeis then loaded into memory, as illustrated in step 208.

Advantageously, rather than loading the entire adjacent stripe intomemory, a smaller subsection of the adjacent stripe can be used instead.For example, in the case of a stripe that comprises 2,000×60,000 pixels,a sub-stripe comprising 40×60,000 pixels can be loaded into memory fromthe adjacent edge of the previously captured adjacent stripe.Additionally, a second sub-stripe comprising 40×60,000 pixels can beloaded into memory from near the adjacent edge of the captured stripe.The two facing 40 pixel wide sub-stripe regions from the two stripesadvantageously provide enough overlap to accurately align the twostripes, which takes place in step 210. This alignment techniqueadvantageously requires significantly less system resources toaccurately align adjacent stripes.

The alignment information generated by this process can be accumulatedfor all stripes that have been aligned and stored in a stripe offsetfile as x-axis and y-axis pixel offsets, as illustrated in step 212. Inone embodiment, the format for each row in the stripe offset file is<filename_n.tif x-offset y-offset> where n is the stripe number,x-offset is the number of pixels by which the adjacent stripes areoffset horizontally, and y-offset is the number of pixels by which theadjacent stripes are offset vertically. FIG. 16 is a block diagramillustrating an example stripe offset file according to an embodiment ofthe present invention. Of course, in an alternative embodiment, thex-offset value can represent the vertical offset while the y-offsetvalue can represent the horizontal offset.

In parallel with aligning the stripes using the sub-stripes, a thumbnailimage is extracted from the stripe. When the various thumbnail imagesfor each stripe in the baseline image are combined, they preferablycreate a thumbnail image for the entire baseline image. Thus, in step214, the thumbnail image for the captured stripe is updated into thethumbnail image file. Preferably, a typical thumbnail image for thebaseline image is 500×300 pixels and can be used by viewing softwarewhich accesses the image data directly from the captured stripe files.

In step 216, after the stripes have been aligned and the thumbnail filehas been updated, the stripe is logically organized into standard imagetiles. These standard image tiles advantageously provide an index intothe large baseline image so that various sub-regions of the baselineimage can be quickly accessed and viewed by the viewing software. Oncethe standard image tiles for a stripe have been identified, the stripecan be written to disk or some other data storage device, as shown instep 218.

Alternatively, the standard image tiles may be written to an open TIFFfile. In such an embodiment, the standard image tiles are preferablycompressed using the JPEG2000 prior to being written to the TIFF file.Additionally, when the standard image files are stored in an open TIFFfile, the native stripe can be discarded rather than being written todisk.

If additional stripes are required to capture the entire baseline image,as determined in step 220, the process then returns to step 200 toacquire the next stripe. If the complete baseline image has beencaptured and no additional stripes are required, then the process iscomplete and terminates as shown in step 222.

There are three outputs of the flow diagram shown in FIG. 15. First, thevarious stripes that comprise the baseline image are stored to disk,preferably in TIFF format and logically organized into standard imagetiles to support efficient viewing. Second, a thumbnail image of thecomplete baseline image, preferably in TIFF format. And third, a stripeoffset file that provides the alignment offsets for the adjacent stripesthat comprise the baseline image.

Alternatively, only a single TIFF file can be created by the process. Insuch an embodiment, the single TIFF file preferably a baseline imagecomprising a plurality of JPEG2000 compressed image tiles. Additionally,the single TIFF file may also include various other images at differentintermediate resolutions and a thumbnail image, which preferablyrepresents the entire baseline image at a low resolution.

Image Data File Organization

There are at least two optional ways to store a virtual slide. First,the virtual slide can be stored as a single TIFF file including abaseline image and one or more intermediate images at differentresolutions, with each image being organized into a plurality of imagetiles. Second, the virtual slide can be stored as a set of image stripesin combination with a stripe offset file that provides physicalarrangement information for aligning the stripes into a contiguousbaseline image.

The storage of image stripes in a single TIFF file including acontiguous baseline image that has been organized into standard imagetiles enables the immediate and efficient viewing of the uncompressedbaseline image as captured by the line-scanner. The dramatic performancedifferences in viewing efficiencies highlighted by the example comparingreading discrete one pixel wide stripes versus reading standard imagetiles illustrates the importance of properly organizing a virtual slidefile. The image data file organization is driven by the desire toefficiently display any user-selected region of the baseline image atany level of zoom (magnification) on a computer monitor.

FIG. 17 is a block diagram illustrating an example viewing platform forvirtual slides according to an embodiment of the present invention. Adisplay image 250 comprising all of the pixels available at a givendisplay resolution is presented on a monitor 252. The display image 250is typically comprised of toolbars, text, and one or more of thefollowing images: a thumbnail image 240, an intermediate zoom image 226and a high resolution image 246 that preferably corresponds to ahigh-resolution region of interest (“ROI”) of a virtual slide.Additionally, the thumbnail image 240 has a thumbnail ROI 242 and theintermediate zoom image 226 has an intermediate zoom image ROI 244.

The thumbnail image 240 represents a very small low-resolution image ofthe entire slide, providing only macroscopic details. The thumbnailimage 240 can be the same thumbnail file that is generated during thepreviously described image assembly process. The intermediate zoom image226 preferably corresponds approximately to what can be observed througha conventional microscope at low optical magnification, typically usingan objective lens with 2× (times two) magnification.

The high resolution image 246 typically corresponds to the nativeresolution of the baseline image, and is intended to correlate with theresolution of imagery data that can be observed through a conventionalmicroscope at high optical magnifications, typically using objectivelenses with 20× or 40× magnification, or higher.

It is clear that there need not be any fixed resolution for any of thesethree images, nor is it necessary that all three images be displayedsimultaneously in the display image 250. There are also many ways toorganize, size and display these and other relevant images to make theviewing of virtual slides on a display monitor as efficient as possible.The challenge for the image file organization is to support suchapplications efficiently. The method of logically indexing a baselineimage (derived from a plurality of aligned stripes) into standard imagetiles makes it dramatically more efficient to access imagery data at thefull resolution of the baseline image.

An image that is logically divided into standard image tiles isrelatively easy to pan at its native 1:1 resolution, as it is necessaryonly to display incremental standard image tiles. However, even with thebenefits of the standard image tiles, panning at lower resolutions thanthe 1:1 resolution of the baseline image is difficult. For example,panning an intermediate zoom image 226 that represents 1/100 the amountof imagery data available in the baseline image is very difficult—evenwhen working with a relatively small number of stripes that have beenorganized into standard image tiles. Panning at this resolution requiresopening a large number of stripes to gain access to the various standardimage tiles of the baseline image that are required to display theproportionately larger area (at lower resolution) of the intermediatezoom image 226.

In this example, 100 times as many standard image tiles must be accessedand sub-sampled to extract the appropriate pixels needed to update theintermediate zoom image 226. The disk access and processing overheadrequired to open the various standard image tiles and sub-sample such alarge number of standard image tiles may result in unacceptableperformance for any viewing software.

FIG. 18 is a block diagram illustrating an example virtual slide imagefile structured to enable efficient viewing according to an embodimentof the present invention. Advantageously, the organization of virtualslides into pyramids with levels of varying resolutions facilitatesrapid zooming and panning by specialized viewing and processingsoftware.

At the base of the pyramid is a baseline image 80 that is logicallydivided into baseline standard image tiles 260. Each baseline standardimage tile 260 represents one 240×240 pixel region of the baseline image80. A pyramid of lower resolution images is created from the baselineimage via straight-line averaging of the pixels in the baseline image.These lower resolution images are themselves logically organized andindexed into intermediate zoom standard image tiles 262 at the same240×240 pixel size. Advantageously, there can be one or more levels ofintermediate zoom images 226 in the pyramid at any desired sub-samplinginterval. At the top of the pyramid is preferably the thumbnail image240, which is not organized into standard image tiles. Preferably, theaspect ratio of the thumbnail image 240 is the same as the aspect ratioof the baseline image 80.

In the illustrated embodiment, the dimension of the thumbnail image 240is preferably 240×240 pixels. Not counting the thumbnail image 240 orthe baseline image 80, the illustrated image pyramid has only twolevels. The first level is the intermediate zoom image 226 and islogically divided into 4×4 intermediate zoom standard image tiles 262,or 960×960 pixels. The second intermediate level is logically dividedinto 2×2 standard image tiles, or 480×480 pixels. The baseline image 80is logically divided into 8×8 standard image tiles 260, or 1,920×1,920pixels. Advantageously, the two intermediate level images respectivelyrepresent sub-sampling ratios of 2:1 and 4:1 with respect to thebaseline image 80 and correspond to an incremental image size that is31.26% larger than the baseline image alone (¼+ 1/16).

The following table summarizes this simplified example.

TABLE 1 Image Ratio Width Height Tile size Raw size Size % Base 1:11,920 1,920 240 × 240 10.8 MB 100.00% level 1 2:1 960 960 240 × 240 2.64MB +25.00% level 2 4:1 480 480 240 × 240 0.66 MB +6.25% Thumb- 8:1 240240 no tiles  .16 MB +.01% nail Total 14.26 MB  +131.26%

The concept of creating lower level pyramids at levels corresponding to2:1, 4:1, 16:1, 32:1, 64:1, etc. will be understood by those havingordinary skill in the art. For example, the well-known flashpix (“FPX”)format utilizes a pyramid format using progressively lower resolutionimages that are based on the JPEG format. The compression achievablewith JPEG is limited to about 10:1 for microscopy images. Given that theflashpix pyramid approach increases the final image files by 33% (¼+⅛+1/16+ 1/32+ 1/64+ . . . . =⅓), the best overall result is anapproximately 8:1 compression. This level of compression is notpractical when dealing with multi-gigabyte images. Additionallimitations of flashpix are that imagery data are only available at thespecific sub-sampling levels, continuous zooming is not supported, andthe maximum file size is limited to 2 GB for the total of all the imagesin the pyramid.

The pyramid approach adopted in one embodiment of the present inventiondiffers from the flashpix approach in the following ways: (1)compression is based on the JPEG 2000 standard; (2) the number ofintermediate levels is greatly reduced; (3) continuous zoom is possible;and (4) virtual slide size is practically unlimited. The following tableillustrates the relative ratios of the intermediate level images thatare created for a virtual slide with a baseline image 80 comprising 56GB of data.

TABLE 2 Image Ratio Width Height Tile size Data size Size % Base  1:1200,000 100,000 240 × 240 57,220 MB 100.00% level 1  4:1 50,000 25,000240 × 240 3,576 MB +6.25% level 2 12:1 16,666 8,333 240 × 240 397.3 MB+.69% level 3 36:1 5,555 2,777 240 × 240 44.13 MB +.08% level 4 108:1 1,851 925 240 × 240 4.89 MB +.01% Thumbnail 391:1  512 256 no tiles .38MB +.00% Total 61,242.7 MB 107.03%

It is noteworthy that the 240×240 pixel standard image tiles size is notarbitrary. The standard image tile size is selected to facilitateinteger sub-sampling of intermediate level images at either 3:1 or 4:1ratios. Another preferred standard image tile size is 256×256 pixels.

JPEG2000 is a standard for image compression which uses wavelettechnology and does not suffer from the block artifacts commonlyobserved in JPG compressed images. JPEG2000 technology involves samplingan image at successively lower and lower frequencies (e.g., powers of2). The frequency data can be used to reconstruct the image at differentresolutions which are down-sampled by powers of 2 from the originalimage. Resolution levels between powers of 2 are synthesized byinterpolating (e.g., down-sampling) from the next larger availablelevel.

In one embodiment, as shown in TABLE 2, the thumbnail ratio representsless than 1% of the baseline image. The spacing between levels is muchlarger than 2:1, which has the benefit of adding only 7% to the size ofthe baseline image. Since all image quadrants are compressed using theJPEG2000 compression standard, much higher compression ratios areachievable. For example, using a 7/9 wavelet filter scheme with aquality of 30, compression ratios of 50:1 have been found to beacceptable for many microscopy images. Adding 7% for the additionalpyramid levels still yields an overall compression of about 45:1.

Furthermore, because each standard image tile 260 is itself a compressedJPEG2000 image each baseline standard image tile 260 may advantageouslyhave its own JPEG2000 pyramid structure available at no additional costin size because of the inherent pyramid structure within JPEG2000. Theinternal pyramid structure of JPEG2000 also makes it possible togenerate intermediate resolution images by interpolating from theclosest level in the pyramid.

Because the pyramid scheme involves multiple layers of images (e.g.,baseline, intermediate, and thumbnail), the preferred file format willallow multiple images to be stored together. The Tagged Image FileFormat (“TIFF”) provides just such a capability. TIFF additionallyprovides other attractive characteristics including: (i) it is anon-proprietary public standard; (ii) it is generally available withefficient open-source implementation (e.g., libtiff); (iii) it supportsvarious image organizations, including standard image tiles; (iv) itsupports various image characteristics such as the number of colorchannels, the bit size of samples in color channels, and color space(RGB, YCC, HSV, etc.); (v) it supports various compression technologies,including those implemented externally from the file access method; (vi)it supports arbitrarily large images; and (vii) it supports storage ofapplication-specific indicative data in image files.

In one embodiment, a TIFF file is used as the file type for a virtualslide. For example, the first image in the TIFF file can be the baselineimage 80 followed by the thumbnail image 240 and then followed by theintermediate level images 226 in the pyramid. There can be more than oneintermediate level image. The baseline image 80 and intermediate levelimages 226 are logically organized into standard image tiles such asbaseline image tiles 260 and each standard image tile is preferablycompressed, for example with JPEG2000.

FIG. 19 is a block diagram illustrating an example image compressor 266for generating a virtual slide 268 according to an embodiment of thepresent invention. The input to the image compressor 266 includes thestripes 222 that have been logically organized into standard image tilesand preferably saved in TIFF format and the stripe offset file 228. Notethat a thumbnail file is not required to create a virtual slide 268.Rather, the image compressor 266 can create a thumbnail image bydown-sampling the baseline image as the virtual slide is created.

In one embodiment, the image compressor 266 is a software program thatis an ActiveX control that is used to create and compress the pyramidlevels that constitute a virtual slide 268 and also to crop, scale,rotate and enhance image files. Specific features of the imagecompressor 266 may include: (i) support for TIFF input files withvarious compression schemes (raw, LZW lossless, JPEG lossy, and JPEG2000lossy) and organizations (raster, stripped, tiled); (ii) support forcompound input files; (iii) ability to generate TIFF output files withvarious compression schemes (raw, LZW lossless, and JPEG2000 lossy) andconfigurable tile size; (iv) ability to optionally generate a thumbnailimage of specified dimensions (stored in output TIFF file as a separateimage); (v) ability to optionally generate one or more intermediateresolution images spaced at certain specified intervals between thebaseline image and the thumbnail image and stored in output TIFF file asthird, fourth, etc. images; (vi) support for large images (e.g., 200,000pixels in width and height); and (vii) support for high-fidelity scalingroutines for down-sampling or up-sampling images to desired dimensions.

With respect to item (ii), a compound file is a text file (e.g., .txt)that describes a mosaic of image files that are combined to form thesource image. Each line in the text file contains an image filename andthe X and Y offsets within the compound image at which the image ispositioned. For example, the stripe offset table shown in FIG. 16 is acompound file.

In another embodiment, the image compressor 266 creates a blank TIFFfile and receives only the standard image tiles from the capturedstripes. These tiles are then placed in the TIFF file and organized intothe baseline image. Additional intermediate zoom level images may alsobe created and placed in the TIFF file along with a thumbnail image thattops off the virtual slide 268.

In order to improve virtual slide processing times, dedicated hardwareto accelerate image processing may be used. In this context hardwarerefers to an external subsystem of the control computer that isdedicated to this processing. In particular, the term hardwarecontemplates the fact that modern hardware actually consists of somecombination of hardware, memory, and software.

Some of the image processing steps may be performed in the line-scancamera interface board/frame grabber as image stripes are acquired. Infact, it is conceivable that all image processing be performed in theframe grabber, automatically yielding compressed standard image tilesfor the baseline and all intermediate level images, as well a thumbnailimage. It is also possible to perform some image processing steps in theframe grabber, perform other image processing steps in the controlcomputer's software, and then use other hardware (not the frame grabber)to perform the final compression, which is the mostcomputation-intensive step. A preferred method of compressing imagerydata from the linear-array-based scanner is a processing board withmodular processing elements, such that, as more processing elements areadded, faster data compression is achieved.

Another embodiment may take advantage of the availability of relativelyinexpensive multiprocessor computers. In this implementation, oneprocessor can be used for data acquisition, interfacing to the capturedevice, and performing adjustment processing such as illuminationcorrection and chromatic aberration correction. A second processor canbe used to perform the image compressor tasks, in parallel with dataacquisition. This embodiment advantageously provides captured stripesdirectly from the acquisition process to the compression andorganization process without intermediate storage to disk. As thediscussion above illustrates, the captured stripes are large,particularly prior to compression, and hence the disk I/O overheadrequired to write and read back these stripes is significant. Such anembodiment would therefore enable access to the fully organized virtualslides more quickly, providing significant benefit to many applications,including applications such as telemicroscopy where it is oftendesirable to quickly share a virtual slide through a network.

Virtual Slide System Components

FIG. 20 is a block diagram illustrating an example system for datamanagement of virtual slides according to an embodiment of the presentinvention. The overall slide scanning system 330 comprises severalcomponents, including a slide scanner 270, a lab console 272, a supportconsole 276, an image analyzer 278, an image viewer client 300, and animage server 320. In the illustrated embodiment, the various componentsare communicatively coupled over a network 312. Alternatively some ofthe components may be combined in a single discrete hardware componentthat houses the combined components. For example, the image server 320and image analyzer 278 may be combined. In such an embodiment, thecombined components may communicate through inter-process communications(e.g., pipes, swap files, etc.) rather than communicate over the network312.

Network 312 may be a local network or LAN, a WAN, wireless network orother type of communication network, including the internet. Preferably,network 312 provides sufficient bandwidth to support the efficient andfast transfer of large amounts of data, such as the data associated withlinear-array-based microscope slide scanner images.

Furthermore, the slide scanner 270 preferably includes a slide scannercontroller program 282. The image server 320 preferably stores virtualslides in a data storage area such as the image files 328, the virtualslides being created by the slide scanner 270. The image viewer client300 is preferably configured to communicate with the image server 320 toallow the remote viewing of virtual slides. Additionally, the laboratoryconsole 272 preferably controls one or more slide scanners 270, forexample through the use of a slide scanner console program 274.Similarly, a support console 276 may provide control for one or moreremote slide scanners 270 using the same slide scanner console program274. Finally, the image analyzer 278 preferably includes an algorithmframework 280 and image analysis software 328 and provides a means foranalyzing, processing and compressing virtual slides. The algorithmframework 280 makes it straightforward to apply traditional imageanalysis software and algorithms to a multi-gigabyte virtual slide.

In one embodiment, the slide scanner 270 preferably has dedicatedcomputer hardware to provide the processing power for scanning amicroscope slide and creating the virtual slide. The other components ofthe slide scanning system 330, namely the image server 320, the labconsole 272, the support console 276, the image analyzer support console276, the image analyzer 278 and the image viewer client 300 can all beintegrated into a single computer, or distributed on multiple computersas needed.

Turning now to FIG. 22, a block diagram of a preferred embodiment of anoptical microscopy system 10 according to the present invention isshown. The heart of the system 10 is a microscope slide scanner 11 thatserves to scan and digitize a specimen or sample 12. The sample 12 canbe anything that may be interrogated by optical microscopy. Forinstance, the sample 12 may be a microscope slide or other sample typethat may be interrogated by optical microscopy. A microscope slide isfrequently used as a viewing substrate for specimens that includetissues and cells, chromosomes, DNA, protein, blood, bone marrow, urine,bacteria, beads, biopsy materials, or any other type of biologicalmaterial or substance that is either dead or alive, stained orunstained, labeled or unlabeled. The sample 12 may also be an array ofany type of DNA or DNA-related material such as cDNA or RNA or proteinthat is deposited on any type of slide or other substrate, including anyand all samples commonly known as a microarray. The sample 12 may be amicrotiter plate, for example a 96-well plate. Other examples of thesample 12 include integrated circuit boards, electrophoresis records,petri dishes, film, semiconductor materials, forensic materials, ormachined parts.

The scanner 11 includes a motorized stage 14, a microscope objectivelens 16, a line scan camera 18, and a data processor 21. The sample 12is positioned on the motorized stage 14 for scanning. The motorizedstage 14 is connected to a stage controller 22 which is connected inturn to the data processor 21. The data processor 21 determines theposition of the sample 12 on the motorized stage 14 via the stagecontroller 22. In the presently preferred embodiment, the motorizedstage 14 moves the sample 12 in at least the two axes (x/y) that are inthe plane of the sample 12. Fine movements of the sample 12 along theoptical z-axis may also be necessary for certain applications of thescanner 11, for example, for focus control. Z-axis movement ispreferably accomplished with a piezo positioner 23, such as the PIFOCfrom Polytec PI or the MIPOS 3 from Piezosystem Jena. The piezopositioner 23 is attached directly to the microscope objective 16 and isconnected to and directed by the data processor 21 via a piezocontroller 27. A means of providing a coarse focus adjustment may alsobe needed and can be provided by z-axis movement as part of themotorized stage 14 or a manual rack-and-pinion coarse focus adjustment(not shown).

In the presently preferred embodiment, the motorized stage 14 includes ahigh precision positioning table with ball bearing linear ways toprovide smooth motion and excellent straight line and flatness accuracy.For example, the motorized stage 14 could include two Daedal model106004 tables stacked one on top of the other. Other types of motorizedstages 14 are also suitable for the scanner 11, including stacked singleaxis stages based on ways other than ball bearings, single- ormultiple-axis positioning stages that are open in the center and areparticularly suitable for trans-illumination from below the sample, orlarger stages that can support a plurality of samples. In the presentlypreferred embodiment, motorized stage 14 includes two stackedsingle-axis positioning tables, each coupled to two millimeterlead-screws and Nema-23 stepping motors. At the maximum lead screw speedof twenty-five revolutions per second, the maximum speed of the sample12 on the motorized stage 14 is fifty millimeters per second. Selectionof a lead screw with larger diameter, for example five millimeters, canincrease the maximum speed to more than 100 millimeters per second. Themotorized stage 14 can be equipped with mechanical or optical positionencoders which has the disadvantage of adding significant expense to thesystem. Consequently, the presently preferred embodiment does notinclude position encoders. However, if one were to use servo motors inplace of stepping motors, then one would have to use position feedbackfor proper control.

Position commands from the data processor 21 are converted to motorcurrent or voltage commands in the stage controller 22. In the presentlypreferred embodiment, the stage controller 22 includes a 2-axisservo/stepper motor controller (Compumotor 6K2) and two 4-ampmicrostepping drives (Compumotor OEMZL4). Microstepping provides a meansfor commanding the stepper motor in much smaller increments than therelatively large single 1.8 degree motor step. For example, at amicrostep of 100, the sample 12 can be commanded to move at steps assmall as 0.1 micrometer. A microstep of 25,000 is used in the presentlypreferred embodiment of this invention. Smaller step sizes are alsopossible. It should be obvious that the optimum selection of themotorized stage 14 and the stage controller 22 depends on many factors,including the nature of the sample 12, the desired time for sampledigitization, and the desired resolution of the resulting digital imageof the sample 12.

The microscope objective lens 16 can be any microscope objective lenscommonly available. One of ordinary skill in the art will realize thatthe choice of which objective lens to use will depend on the particularcircumstances. In the preferred embodiment of the present invention, themicroscope objective lens 16 is of the infinity-corrected type.

The sample 12 is illuminated by an illumination system 29 that includesa light source 31 and illumination optics 32. The light source 31 in thepresently preferred embodiment includes a variable intensity halogenlight source with a concave reflective mirror to maximize light outputand a KG-1 filter to suppress heat. However, the light source 31 couldalso be any other type of arc-lamp, laser, or other source of light. Theillumination optics 32 in the presently preferred embodiment include astandard Köhler illumination system with two conjugate planes that areorthogonal to the optical axis. The illumination optics 32 arerepresentative of the bright-field illumination optics that can be foundon most commercially available compound microscopes sold by companiessuch as Carl Zeiss, Nikon, Olympus, or Leica. One set of conjugateplanes includes (i) a field iris aperture illuminated by the lightsource 31, (ii) the object plane that is defined by the focal plane ofthe sample 12, and (iii) the plane containing the light-responsiveelements of the line scan camera 18. A second conjugate plane includes(i) the filament of the bulb that is part of the light source 31, (ii)the aperture of a condenser iris that sits immediately before thecondenser optics that are part of the illumination optics 32, and (iii)the back focal plane of the microscope objective lens 16. In thepresently preferred embodiment, the sample 12 is illuminated and imagedin transmission mode, with the line scan camera 18 sensing opticalenergy that is transmitted by the sample 12, or conversely, opticalenergy that is absorbed by the sample 12.

The scanner 11 of the present invention is equally suitable fordetecting optical energy that is reflected from the sample 12, in whichcase the light source 31, the illumination optics 32, and the microscopeobjective lens 16 must be selected based on compatibility withreflection imaging. One possible embodiment may therefore beillumination through a fiber optic bundle that is positioned above thesample 12. Other possibilities include excitation that is spectrallyconditioned by a monochromator. If the microscope objective lens 16 isselected to be compatible with phase-contrast microscopy, then theincorporation of at least one phase stop in the condenser optics thatare part of the illumination optics 32 will enable the scanner 11 to beused for phase contrast microscopy. To one of ordinary skill in the art,the modifications required for other types of microscopy such asdifferential interference contrast and confocal microscopy should bereadily apparent. Overall, the scanner 11 is suitable, with appropriatebut well-known modifications, for the interrogation of microscopicsamples in any known mode of optical microscopy.

Between the microscope objective lens 16 and the line scan camera 18 aresituated the line scan camera focusing optics 33 that focus the opticalsignal captured by the microscope objective lens 16 onto thelight-responsive elements of the line scan camera 18. In a moderninfinity-corrected microscope the focusing optics between the microscopeobjective lens and the eyepiece optics, or between the microscopeobjective lens and an external imaging port, consist of an opticalelement known as a tube lens that is part of a microscope's observationtube. Many times the tube lens consists of multiple optical elements toprevent the introduction of coma or astigmatism. One of the motivationsfor the relatively recent change from traditional finite tube lengthoptics to infinity corrected optics was to increase the physical spacein which the optical energy from the sample 12 is parallel, meaning thatthe focal point of this optical energy is at infinity. In this case,accessory elements like dichroic mirrors or filters can be inserted intothe infinity space without changing the optical path magnification orintroducing undesirable optical artifacts.

Infinity-corrected microscope objective lenses are typically inscribedwith an infinity mark. The magnification of an infinity correctedmicroscope objective lens is given by the quotient of the focal lengthof the tube lens divided by the focal length of the objective lens. Forexample, a tube lens with a focal length of 180 millimeters will resultin 20× magnification if an objective lens with 9 millimeter focal lengthis used. One of the reasons that the objective lenses manufactured bydifferent microscope manufacturers are not compatible is because of alack of standardization in the tube lens focal length. For example, a20× objective lens from Olympus, a company that uses a 180 millimetertube lens focal length, will not provide a 20× magnification on a Nikonmicroscope that is based on a different tube length focal length of 200millimeters. Instead, the effective magnification of such an Olympusobjective lens engraved with 20× and having a 9 millimeter focal lengthwill be 22.2×, obtained by dividing the 200 millimeter tube lens focallength by the 9 millimeter focal length of the objective lens. Changingthe tube lens on a conventional microscope is virtually impossiblewithout disassembling the microscope. The tube lens is part of acritical fixed element of the microscope. Another contributing factor tothe incompatibility between the objective lenses and microscopesmanufactured by different manufacturers is the design of the eyepieceoptics, the binoculars through which the specimen is observed. Whilemost of the optical corrections have been designed into the microscopeobjective lens, most microscope users remain convinced that there issome benefit in matching one manufacturers' binocular optics with thatsame manufacturers' microscope objective lenses to achieve the bestvisual image.

The line scan camera focusing optics 33 include a tube lens opticmounted inside of a mechanical tube. Since the scanner 11, in itspreferred embodiment, lacks binoculars or eyepieces for traditionalvisual observation, the problem suffered by conventional microscopes ofpotential incompatibility between objective lenses and binoculars isimmediately eliminated. One of ordinary skill will similarly realizethat the problem of achieving parfocality between the eyepieces of themicroscope and a digital image on a display monitor is also eliminatedby virtue of not having any eyepieces. Since the scanner 11 alsoovercomes the field of view limitation of a traditional microscope byproviding a field of view that is practically limited only by thephysical boundaries of the sample 12, the importance of magnification inan all-digital imaging microscope such as provided by the presentscanner 11 is limited. Once a portion of the sample 12 has beendigitized, it is straightforward to apply electronic magnification,sometimes known as electric zoom, to an image of the sample 12 in orderto increase its magnification. Increasing the magnification of an imageelectronically has the effect of increasing the size of that image onthe monitor that is used to display the image. If too much electroniczoom is applied, then the display monitor will be able to show onlyportions of the magnified image. It is not possible, however, to useelectronic magnification to display information that was not present inthe original optical signal that was digitized in the first place. Sinceone of the objectives of the scanner 11 is to provide high qualitydigital images, in lieu of visual observation through the eyepieces of amicroscope, it is important that the content of the images acquired bythe scanner 11 include as much image detail as possible. The termresolution is typically used to describe such image detail and the termdiffraction-limited is used to describe the wavelength-limited maximumspatial detail available in an optical signal. The scanner 11 providesdiffraction-limited digital imaging by selection of a tube lens focallength that is matched according to the well know Nyquist samplingcriteria to both the size of an individual pixel element in alight-sensing camera such as the line scan camera 18 and to thenumerical aperture of the microscope objective lens 16. It is well knownthat numerical aperture, not magnification, is the resolution-limitingattribute of a microscope objective lens 16.

An example will help to illustrate the optimum selection of a tube lensfocal length that is part of the line scan camera focusing optics 33.Consider again the 20× microscope objective lens 16 with 9 millimeterfocal length discussed previously and assume that this objective lenshas a numerical aperture of 0.50. Assuming no appreciable degradationfrom the condenser, the diffraction-limited resolving power of thisobjective lens at a wavelength of 500 nanometers is approximately 0.6micrometers, obtained using the well-known Abbe relationship. Assumefurther that the line scan camera 18, which in its preferred embodimenthas a plurality of 14 micrometer square pixels, is used to detect aportion of the sample 12. In accordance with sampling theory, it isnecessary that at least two sensor pixels subtend the smallestresolvable spatial feature. In this case, the tube lens must be selectedto achieve a magnification of 46.7, obtained by dividing 28 micrometers,which corresponds to two 14 micrometer pixels, by 0.6 micrometers, thesmallest resolvable feature dimension. The optimum tube lens optic focallength is therefore about 420 millimeters, obtained by multiplying 46.7by 9. The line scan focusing optics 33 with a tube lens optic having afocal length of 420 millimeters will therefore be capable of acquiringimages with the best possible spatial resolution, similar to what wouldbe observed by viewing a specimen under a microscope using the same 20×objective lens. To reiterate, the scanner 11 utilizes a traditional 20×microscope objective lens 16 in a higher magnification opticalconfiguration, in this example about 47×, in order to acquirediffraction-limited digital images. If a traditional 20× magnificationobjective lens 16 with a higher numerical aperture were used, say 0.75,the required tube lens optic magnification for diffraction-limitedimaging would be about 615 millimeters, corresponding to an overalloptical magnification of 68×. Similarly, if the numerical aperture ofthe 20× objective lens were only 0.3, the optimum tube lens opticmagnification would only be about 28×, which corresponds to a tube lensoptic focal length of approximately 252 millimeters. The line scancamera focusing optics 33 are modular elements of the scanner 11 and canbe interchanged as necessary for optimum digital imaging. The advantageof diffraction-limited digital imaging is particularly significant forapplications, for example bright field microscopy, in which thereduction in signal brightness that accompanies increases inmagnification is readily compensated by increasing the intensity of anappropriately designed illumination system 29.

In principle, it is possible to attach external magnification-increasingoptics to a conventional microscope-based digital imaging system toeffectively increase the tube lens magnification so as to achievediffraction-limited imaging as has just been described for the presentscanner 11; however, the resulting decrease in the field of view isoften unacceptable, making this approach impractical. Furthermore, manyusers of microscopes typically do not understand enough about thedetails of diffraction-limited imaging to effectively employ thesetechniques on their own. In practice, digital cameras are attached tomicroscope ports with magnification-decreasing optical couplers toattempt to increase the size of the field of view to something moresimilar to what can be seen through the eyepiece. The standard practiceof adding de-magnifying optics is a step in the wrong direction if thegoal is to obtain diffraction-limited digital images.

In a conventional microscope, different power objectives lenses aretypically used to view the specimen at different resolutions andmagnifications. Standard microscopes have a nosepiece that holds fiveobjectives lenses. In an all-digital imaging system such as the presentscanner 11 there is a need for only one microscope objective lens 16with a numerical aperture corresponding to the highest spatialresolution desirable. The presently preferred embodiment of the scanner11 provides for only one microscope objective lens 16. Once adiffraction-limited digital image has been captured at this resolution,it is straightforward using standard digital image processingtechniques, to present imagery information at any desirable reducedresolutions and magnifications.

The presently preferred embodiment of the scanner 11 is based on a DalsaSPARK line scan camera 18 with 1024 pixels (picture elements) arrangedin a linear array, with each pixel having a dimension of 14 by 14micrometers. Any other type of linear array, whether packaged as part ofa camera or custom-integrated into an imaging electronic module, canalso be used. The linear array in the presently preferred embodimenteffectively provides eight bits of quantization, but other arraysproviding higher or lower level of quantization may also be used.Alternate arrays based on 3-channel red-green-blue (RGB) colorinformation or time delay integration (TDI), may also be used. TDIarrays provide a substantially better signal-to-noise ratio (SNR) in theoutput signal by summing intensity data from previously imaged regionsof a specimen, yielding an increase in the SNR that is in proportion tothe square-root of the number of integration stages. TDI arrays cancomprise multiple stages of linear arrays. TDI arrays are available with24, 32, 48, 64, 96, or even more stages. The scanner 11 also supportslinear arrays that are manufactured in a variety of formats includingsome with 512 pixels, some with 1024 pixels, and others having as manyas 4096 pixels. Appropriate, but well known, modifications to theillumination system 29 and the line scan camera focusing optics 33 maybe required to accommodate larger arrays. Linear arrays with a varietyof pixel sizes can also be used in scanner 11. The salient requirementfor the selection of any type of line scan camera 18 is that the sample12 can be in motion with respect to the line scan camera 18 during thedigitization of the sample 12 in order to obtain high quality images,overcoming the static requirements of the conventional imaging tilingapproaches known in the prior art.

The output signal of the line scan camera 18 is connected to the dataprocessor 21. The data processor 21 in the presently preferredembodiment includes a central processing unit with ancillaryelectronics, for example a motherboard, to support at least one signaldigitizing electronics board such as an imaging board or a framegrabber. In the presently preferred embodiment, the imaging board is anEPIX PIXCID24 PCI bus imaging board, however, there are many other typesof imaging boards or frame grabbers from a variety of manufacturerswhich could be used in place of the EPIX board. An alternate embodimentcould be a line scan camera that uses an interface such as IEEE 1394,also known as Firewire, to bypass the imaging board altogether and storedata directly on a data storage 38, such as a hard disk.

The data processor 21 is also connected to a memory 36, such as randomaccess memory (RAM), for the short-term storage of data, and to the datastorage 38, such as a hard drive, for long-term data storage. Further,the data processor 21 is connected to a communications port 39 that isconnected to a network 41 such as a local area network (LAN), a widearea network (WAN), a metropolitan area network (MAN), an intranet, anextranet, or the global Internet. The memory 36 and the data storage 38are also connected to each other. The data processor 21 is also capableof executing computer programs, in the form of software, to controlcritical elements of the scanner 11 such as the line scan camera 18 andthe stage controller 22, or for a variety of image-processing functions,image-analysis functions, or networking. The data processor 21 can bebased on any operating system, including operating systems such asWindows, Linux, OS/2, Mac OS, and Unix. In the presently preferredembodiment, the data processor 21 operates based on the Windows NToperating system.

The data processor 21, memory 36, data storage 38, and communicationport 39 are each elements that can be found in a conventional computer.One example would be a personal computer such as a Dell Dimension XPST500 that features a Pentium III 500 MHz processor and up to 756megabytes (MB) of RAM. In the presently preferred embodiment, thecomputer, elements which include the data processor 21, memory 36, datastorage 38, and communications port 39 are all internal to the scanner11, so that the only connection of the scanner 11 to the other elementsof the system 10 is the communication port 39. In an alternateembodiment of the scanner 11, the computer elements would be external tothe scanner 11 with a corresponding connection between the computerelements and the scanner 11.

The scanner 11, in the presently preferred embodiment of the invention,integrates optical microscopy, digital imaging, motorized samplepositioning, computing, and network-based communications into asingle-enclosure unit. The major advantage of packaging the scanner 11as a single-enclosure unit with the communications port 39 as theprimary means of data input and output are reduced complexity andincreased reliability. The various elements of the scanner 11 areoptimized to work together, in sharp contrast to traditionalmicroscope-based imaging systems in which the microscope, light source,motorized stage, camera, and computer are typically provided bydifferent vendors and require substantial integration and maintenance.

The communication port 39 provides a means for rapid communications withthe other elements of the system 10, including the network 41. Thepresently preferred communications protocol for the communications port39 is a carrier-sense multiple-access collision detection protocol suchas Ethernet, together with the TCP/IP protocol for transmission controland internetworking. The scanner 11 is intended to work with any type oftransmission media, including broadband, baseband, coaxial cable,twisted pair, fiber optics, DSL or wireless.

In the presently preferred embodiment, control of the scanner 11 andreview of the imagery data captured by the scanner 11 are performed on acomputer 43 that is connected to the network 41. The computer 43, in itspresently preferred embodiment, is connected to a display monitor 47 toprovide imagery information to an operator. A plurality of computers 43may be connected to the network 41. In the presently preferredembodiment, the computer 43 communicates with the scanner 11 using anetwork browser such as Internet Explorer from Microsoft or NetscapeCommunicator from AOL. Images are stored on the scanner 11 in a commoncompressed format such a JPEG which is an image format that iscompatible with standard image-decompression methods that are alreadybuilt into most commercial browsers. Other standard or non-standard,lossy or lossless, image compression formats will also work. In thepresently preferred embodiment, the scanner 11 is a webserver providingan operator interface that is based on webpages that are sent from thescanner 11 to the computer 43. For dynamic review of imagery data, thecurrently preferred embodiment of the scanner 11 is based on playingback, for review on the display monitor 47 that is connected to thecomputer 43, multiple frames of imagery data using standardmultiple-frame browser compatible software packages such as Media-Playerfrom Microsoft, Quicktime from Apple Computer, or RealPlayer from RealNetworks. In the presently preferred embodiment, the browser on thecomputer 43 uses the hypertext transmission protocol (http) togetherwith TCP for transmission control.

There are, and will be in the future, many different means and protocolsby which the scanner 11 could communicate with the computer 43, or aplurality of computers. While the presently preferred embodiment isbased on standard means and protocols, the approach of developing one ormultiple customized software modules known as applets is equallyfeasible and may be desirable for selected future applications of thescanner 11. Further, there are no constraints that computer 43 be of anyspecific type such as a personal computer (PC) or be manufactured by anyspecific company such as Dell. One of the advantages of a standardizedcommunications port 39 is that any type of computer 43 operating commonnetwork browser software can communicate with the scanner 11.

If one so desires, it is possible, with some modifications to thescanner 11, to obtain spectrally resolved images. Spectrally resolvedimages are images in which spectral information is measured at everyimage pixel. Spectrally resolved images could be obtained by replacingthe line scan camera 18 of the scanner 11 with an optical slit and animaging spectrograph. The imaging spectrograph uses a two-dimensionalCCD detector to capture wavelength-specific intensity data for a columnof image pixels by using a prism or grating to disperse the opticalsignal that is focused on the optical slit along each of the rows of thedetector.

The capabilities of the line scan camera 18 typically determine whetherscanning can be done bi-directionally, as in the currently preferredembodiment of the scanner 11, or uni-directionally. Uni-directionalsystems often comprise more than one linear array 73, such as a threechannel color array 86 or a multi-channel TDI array 88 shown in FIG. 23.The color array 86 detects the RGB intensities required for obtaining acolor image. An alternate embodiment for obtaining color informationuses a prism to split the broadband optical signal into the three colorchannels. The TDI array 88 could be used in an alternate embodiment ofthe scanner 11 to provide a means of increasing the effectiveintegration time of the line scan camera 18, while maintaining a fastdata rate, and without significant loss in the signal-to-noise ratio ofthe digital imagery data.

The scanner 11 can be further optimized to minimize the totalacquisition time of the image 76 even more. The image acquisition timethat can be achieved by the scanner 11 depends in part on the line rateof the line scan camera 18. At the line rate of 27,600 lines per secondof the present example, each line image is captured in about 0.04milliseconds. Illumination from the light source that includes a 50 wattbulb, provides sufficient light to register a signal with sufficientsignal-to-noise ratio on the line scan camera. At faster read-out rates,the exposure time per line is reduced and improvements and enhancementsto the illumination system 28 of the scanner 11 may be required.Similarly, for applications of the scanner 11 in which less light isavailable, for example fluorescence, the effective line integration timemust be increased. A TDI type of line scan camera provides an excellentmeans of increasing the effective integration time while maintaining afast data read-out, without significant loss in the signal-to-noiseratio of the imagery data.

FIG. 21 is a block diagram illustrating an exemplary computer system 550that may be used in connection with the various embodiments describedherein. For example, the computer system 550 may be used in conjunctionwith the linear-array-based microscope slide scanner, an image server, alab console or support console, an image analyzer, or an image viewerclient. The computer system 550 may also be used as a separate system toperform certain computationally intense procedures or steps, for examplecompression of the virtual slides. However, other computer systemsand/or architectures may be used, as will be clear to those skilled inthe art.

The computer system 550 preferably includes one or more processors, suchas processor 552. Additional processors may be provided, such as anauxiliary processor to manage input/output, an auxiliary processor toperform floating point mathematical operations, a special-purposemicroprocessor having an architecture suitable for fast execution ofsignal processing algorithms (e.g., digital signal processor), a slaveprocessor subordinate to the main processing system (e.g., back-endprocessor), an additional microprocessor or controller for dual ormultiple processor systems, or a coprocessor. Such auxiliary processorsmay be discrete processors or may be integrated with the processor 552.

The processor 552 is preferably connected to a communication bus 554.The communication bus 554 may include a data channel for facilitatinginformation transfer between storage and other peripheral components ofthe computer system 550. The communication bus 554 further may provide aset of signals used for communication with the processor 552, includinga data bus, address bus, and control bus (not shown). The communicationbus 554 may comprise any standard or non-standard bus architecture suchas, for example, bus architectures compliant with industry standardarchitecture (“ISA”), extended industry standard architecture (“EISA”),Micro Channel Architecture (“MCA”), peripheral component interconnect(“PCI”) local bus, or standards promulgated by the Institute ofElectrical and Electronics Engineers (“IEEE”) including IEEE 488general-purpose interface bus (“GPIB”), IEEE 696/S-100, and the like.

Computer system 550 preferably includes a main memory 556 and may alsoinclude a secondary memory 558. The main memory 556 provides storage ofinstructions and data for programs executing on the processor 552. Themain memory 556 is typically semiconductor-based memory such as dynamicrandom access memory (“DRAM”) and/or static random access memory(“SRAM”). Other semiconductor-based memory types include, for example,synchronous dynamic random access memory (“SDRAM”), Rambus dynamicrandom access memory (“RDRAM”), ferroelectric random access memory(“FRAM”), and the like, including read only memory (“ROM”).

The secondary memory 558 may optionally include a hard disk drive 560and/or a removable storage drive 562, for example a floppy disk drive, amagnetic tape drive, a compact disc (“CD”) drive, a digital versatiledisc (“DVD”) drive, etc. The removable storage drive 562 reads fromand/or writes to a removable storage medium 564 in a well-known manner.Removable storage medium 564 may be, for example, a floppy disk,magnetic tape, CD, DVD, etc.

The removable storage medium 564 is preferably a computer readablemedium having stored thereon computer executable code (i.e., software)and/or data. The computer software or data stored on the removablestorage medium 564 is read into the computer system 550 as electricalcommunication signals 578.

In alternative embodiments, secondary memory 558 may include othersimilar means for allowing computer programs or other data orinstructions to be loaded into the computer system 550. Such means mayinclude, for example, an external storage medium 572 and an interface570. Examples of external storage medium 572 may include an externalhard disk drive or an external optical drive, or and externalmagneto-optical drive.

Other examples of secondary memory 558 may include semiconductor-basedmemory such as programmable read-only memory (“PROM”), erasableprogrammable read-only memory (“EPROM”), electrically erasable read-onlymemory (“EEPROM”), or flash memory (block oriented memory similar toEEPROM). Also included are any other removable storage units 572 andinterfaces 570, which allow software and data to be transferred from theremovable storage unit 572 to the computer system 550.

Computer system 550 may also include a communication interface 574. Thecommunication interface 574 allows software and data to be transferredbetween computer system 550 and external devices (e.g. printers),networks, or information sources. For example, computer software orexecutable code may be transferred to computer system 550 from a networkserver via communication interface 574. Examples of communicationinterface 574 include a modem, a network interface card (“NIC”), acommunications port, a PCMCIA slot and card, an infrared interface, andan IEEE 1394 fire-wire, just to name a few.

Communication interface 574 preferably implements industry promulgatedprotocol standards, such as Ethernet IEEE 802 standards, Fiber Channel,digital subscriber line (“DSL”), asynchronous digital subscriber line(“ADSL”), frame relay, asynchronous transfer mode (“ATM”), integrateddigital services network (“ISDN”), personal communications services(“PCS”), transmission control protocol/Internet protocol (“TCP/IP”),serial line Internet protocol/point to point protocol (“SLIP/PPP”), andso on, but may also implement customized or non-standard interfaceprotocols as well.

Software and data transferred via communication interface 574 aregenerally in the form of electrical communication signals 578. Thesesignals 578 are preferably provided to communication interface 574 via acommunication channel 576. Communication channel 576 carries signals 578and can be implemented using a variety of communication means includingwire or cable, fiber optics, conventional phone line, cellular phonelink, radio frequency (“RF”) link, or infrared link, just to name a few.

Computer executable code (i.e., computer programs or software) is storedin the main memory 556 and/or the secondary memory 558. Computerprograms can also be received via communication interface 574 and storedin the main memory 556 and/or the secondary memory 558. Such computerprograms, when executed, enable the computer system 550 to perform thevarious functions of the present invention as previously described.

In this description, the term “computer readable medium” is used torefer to any media used to provide computer executable code (e.g.,software and computer programs) to the computer system 550. Examples ofthese media include main memory 556, secondary memory 558 (includinghard disk drive 560, removable storage medium 564, and external storagemedium 572), and any peripheral device communicatively coupled withcommunication interface 574 (including a network information server orother network device). These computer readable mediums are means forproviding executable code, programming instructions, and software to thecomputer system 550.

In an embodiment that is implemented using software, the software may bestored on a computer readable medium and loaded into computer system 550by way of removable storage drive 562, interface 570, or communicationinterface 574. In such an embodiment, the software is loaded into thecomputer system 550 in the form of electrical communication signals 578.The software, when executed by the processor 552, preferably causes theprocessor 552 to perform the inventive features and functions previouslydescribed herein.

Various embodiments may also be implemented primarily in hardware using,for example, components such as application specific integrated circuits(“ASICs”), or field programmable gate arrays (“FPGAs”). Implementationof a hardware state machine capable of performing the functionsdescribed herein will also be apparent to those skilled in the relevantart. Various embodiments may also be implemented using a combination ofboth hardware and software.

While the particular systems and methods for data management in alinear-array-based microscope slide scanner herein shown and describedin detail is fully capable of attaining the above described objects ofthis invention, it is to be understood that the description and drawingspresented herein represent a presently preferred embodiment of theinvention and are therefore representative of the subject matter whichis broadly contemplated by the present invention. It is furtherunderstood that the scope of the present invention fully encompassesother embodiments that may become obvious to those skilled in the artand that the scope of the present invention is accordingly limited bynothing other than the appended claims.

What is claimed is:
 1. A system for capturing image data using a linescan camera, the system comprising: an objective lens positioned forviewing at least a portion of a sample during constant relative motion;a line scan camera optically coupled to the objective lens andconfigured to capture image data of the sample as a plurality of imagestripes; and at least one processor configured to, while the line scancamera is capturing at least one of the plurality of image stripes,coarsely align two or more of the plurality of image stripes accordingto a synchronization process, and, subsequently, finely align the two ormore image stripes using pattern matching.
 2. The system of claim 1,further comprising a strobe light, wherein the synchronization processcomprises: for each of the two or more image stripes, capturing an areacomprising an area preceding a predetermined scan area, the scan area,and an area following the scan area, and, during capture by the linescan camera of one or more lines of image data corresponding to abeginning of the scan area, using the strobe light to saturate the oneor more lines of image data; and, aligning the two or more image stripesbased on the saturated one or more lines of image data in each of thetwo or more image stripes.
 3. The system of claim 2, wherein the atleast one processor, prior to storing each of the two or more imagestripes, eliminates, from each image stripe, image data corresponding tothe area preceding the scan area and the area following the scan area.4. The system of claim 2, wherein the saturated one or more lines ofimage data comprise a plurality of lines of image data.
 5. The system ofclaim 1, further comprising: a position encoder that indicates aposition of the sample; and a trigger to start and stop capturing ofimage data by the line scan camera; wherein the synchronization processcomprises, when the position encoder indicates that the position of thesample corresponds to a beginning of a predetermined scan area,controlling the trigger to start capture of image data by the line scancamera, and, when the position encoder indicates that the position ofthe sample corresponds to an end of the scan area, controlling thetrigger to stop capture of image data by the line scan camera.
 6. Thesystem of claim 1, wherein the pattern matching comprises testing aplurality of possible X-Y offset values within overlapping regions ofadjacent ones of the two or more image stripes to determine an X-Yoffset pair having a maximum correlation, wherein the X-Y offset paircomprises an X-Y offset for a first one of the adjacent image stripesand an X-Y offset for a second one of the adjacent image stripes.
 7. Thesystem of claim 6, wherein the pattern matching further comprisesselecting the plurality of possible X-Y offset values by selecting edgepixels within the overlapping regions of the adjacent image stripes. 8.The system of claim 7, wherein selecting edge pixels within theoverlapping regions of the adjacent image stripes comprises: sortingpixels in the overlapping regions according to intensity gradientvalues; and selecting a subset of the sorted pixels having highestintensity gradient values as the edge pixels.
 9. The system of claim 6,wherein the plurality of possible X-Y offset values comprise a range ofX values that corresponds to a coarse alignment uncertainty representinga data latency within the synchronization process.
 10. The system ofclaim 9, further comprising a mechanical stage configured to move thesample relative to the objective lens, wherein the plurality of possibleX-Y offset values comprise a range of Y values that corresponds to amechanical motion uncertainty of the mechanical stage.
 11. A method forcapturing image data using a line scan camera, the method comprising: bya line scan camera, capturing image data of a sample as a plurality ofimage stripes; and, by at least one processor, while the line scancamera is capturing at least one of the plurality of image stripes,coarsely aligning two or more of the plurality of image stripesaccording to a synchronization process, and, subsequently, finelyaligning the two or more image stripes using pattern matching.
 12. Themethod of claim 1, wherein the synchronization process comprises: foreach of the two or more image stripes, capturing an area comprising anarea preceding a predetermined scan area, the scan area, and an areafollowing the scan area, and, during capture by the line scan camera ofone or more lines of image data corresponding to a beginning of the scanarea, using a strobe light to saturate the one or more lines of imagedata; and, aligning the two or more image stripes based on the saturatedone or more lines of image data in each of the two or more imagestripes.
 13. The method of claim 12, further comprising: eliminating,from each image stripe, image data corresponding to the area precedingthe scan area and the area following the scan area; and storing each ofthe two or more image stripes without the eliminated image data.
 14. Themethod of claim 12, wherein the saturated one or more lines of imagedata comprise a plurality of lines of image data.
 15. The method ofclaim 11, wherein the synchronization process comprises: by a positionencoder, indicating a position of the sample; when the position encoderindicates that the position of the sample corresponds to a beginning ofa predetermined scan area, controlling a trigger to start capture ofimage data by the line scan camera, and, when the position encoderindicates that the position of the sample corresponds to an end of thescan area, controlling the trigger to stop capture of image data by theline scan camera.
 16. The method of claim 11, wherein the patternmatching comprises testing a plurality of possible X-Y offset valueswithin overlapping regions of adjacent ones of the two or more imagestripes to determine an X-Y offset pair having a maximum correlation,wherein the X-Y offset pair comprises an X-Y offset for a first one ofthe adjacent image stripes and an X-Y offset for a second one of theadjacent image stripes.
 17. The method of claim 16, wherein the patternmatching further comprises selecting the plurality of possible X-Yoffset values by selecting edge pixels within the overlapping regions ofthe adjacent image stripes.
 18. The method of claim 17, whereinselecting edge pixels within the overlapping regions of the adjacentimage stripes comprises: sorting pixels in the overlapping regionsaccording to intensity gradient values; and selecting a subset of thesorted pixels having highest intensity gradient values as the edgepixels.
 19. The method of claim 16, wherein the plurality of possibleX-Y offset values comprise a range of X values that corresponds to acoarse alignment uncertainty representing a data latency within thesynchronization process.
 20. The method of claim 19, wherein theplurality of possible X-Y offset values comprise a range of Y valuesthat corresponds to a mechanical motion uncertainty of a mechanicalstage configured to move the sample.